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Abstract  -  The  VIIRS  sensor  provides  measurements  for  22 
Environmental  Data  Records  (EDRs)  addressing  the 
atmosphere,  ocean  surface  temperature,  ocean  color,  land 
parameters,  aerosols,  imaging  for  clouds  and  ice,  and  more. 
That  is,  the  VIIRS  collects  visible  and  infrared  radiometric 
data  of  the  Earth's  atmosphere,  ocean,  and  land  surfaces. 
Data  types  include  atmospheric,  clouds,  Earth  radiation 
budget,  land/water  and  sea  surface  temperature,  ocean  color, 
and  low  light  imagery.  This  wide  scope  of  measurements  calls 
for  the  preparation  of  a  multiplicity  of  Algorithm  Theoretical 
Basis  Documents  (ATBDs),  and,  additionally,  for  intermediate 
products  such  as  cloud  mask,  et  al.  Furthermore,  the  VIIRS 
interacts  with  three  or  more  other  sensors.  This  paper 
addresses  selected  and  crucial  elements  of  the  process  being 
used  to  convert  and  test  an  immense  volume  of  a  maturing  and 
changing  science  code  to  the  initial  operational  source  code  in 
preparation  for  launch  of  NPP.  The  integrity  of  the  original 
science  code  is  maintained  and  enhanced  via  baseline 
comparisons  when  re-hosted,  in  addition  to  multiple  planned 
code  performance  reviews. 

I.  Introduction  And  Background 

The  National  Polar-orbiting  Operational  Environmental 
Satellite  System  includes  a  constellation  of  three  satellites, 
associated  telemetry  stations,  and  four  ground  processing 
elements.  These  physical  facilities  are  supported  by 
software  and  activities  which  enable  the  conversion  in  near 
real-time  of  raw  telemetry  data  from  spacecraft  instruments 
to  produce  Sensor  Data  Records  (SDRs)1,  Environmental 
Data  Records  (EDRs)2,  as  well  as  additional  outputs 
including  Intermediate  Products  and  Quality  Flags.  The 


1  Sensor  Data  Records  (SDRs)  are  full  resolution  sensor  data  that  are  time 
referenced,  Earth  located,  and  calibrated  by  applying  the  ancillary 
information,  including  radiometric  and  geometric  calibration  coefficients 
and  geo-referencing  parameters  such  as  platform  ephemeris. 

2  EDRs  are  geophysical  parameters  that  range  from  atmospheric  products 
detailing  cloud  coverage,  temperature,  humidity,  and  ozone  distribution;  to 
land  surface  products  showing  snow  cover,  vegetation,  and  land  use;  to 
ocean  products  depicting  sea  surface  temperatures,  sea  ice,  and  wave 
height;  to  characterizations  of  the  space  environment. 


VIIRS  sensor  is  manifested  on  all  three  of  these  orbits.  This 
paper  discusses  selected  aspects  of  the  process  of  converting 
science  algorithms  to  operational  use  for  a  single  sensor, 
VIIRS.  The  VIIRS  operational  algorithms  are  capable  of 
calculating  and  delivering  environmental  data  records  in 
near  real-time.  The  process  is  applicable  to  both  NPOESS 
and  the  NPOESS  Preparatory  Project  (NPP). 

The  algorithms  used  to  create  these  data  records  are 
delivered  as  science  grade  software  developed  in 
conjunction  with  the  spacecraft  instrument.  The  algorithm’s 
theoretical  foundation  is  described  in  associated  Algorithm 
Theoretical  Basis  Documents  (ATBDs).  Test  data  are 
delivered  by  the  algorithm  subcontractor  to  the  prime, 
Northrop  Grumman  Space  Technology  (NGST).  NGST 
then  re-hosts  the  code  and  verifies  the  test  data  and  results 
provided  by  the  algorithm  vendor,  recording  performance 
metrics  of  the  algorithm  and  identifying  potential  follow-on 
work.  NGST  also  prepares  a  draft  version  of  an  Operational 
Algorithm  Document  (OAD)  specifying  additional  Quality 
Flags  and  Threshold  Tables  for  the  algorithm’.  This 
package,  including  the  algorithm  code,  the  ATBD,  draft 
Operational  Algorithm  Document  (OAD),  test  data,  and  any 
additional  algorithm  subcontractor  documentation  is  passed 
to  Raytheon  for  conversion  to  operational  code.  The 
transfer  of  these  algorithms  and  data  is  performed  via  a 
multi-stage,  boarded  review  process  involving  the 
subcontractor  algorithm  lead,  NGST  algorithm  lead, 
Raytheon  operational  algorithm  lead,  and  general  science 
and  architecture  lead.  Members  of  the  science  community 
and  the  Integrated  Program  Office  are  also  invited. 

The  conversion  process  for  all  of  the  NPOESS  sensors’ 
algorithms  to  operational  code  follows  five  steps,  including 
(1)  Porting  the  algorithm  code  to  an  IBM  AIX  (Unix) 


3  The  OAD  is  a  description  of  the  computer  science  implementation  of  the 
algorithm,  using  software  architecture,  pseudo  code  and/or  flowcharts. 
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platform4,  (2)  Design  and  implementation  of  distinguishable 
Input,  Processing,  and  Output  processes,  making  use  of 
standard  I/O  library  functions,  (3)  Implementation  of  error 
handling  and  data  quality  reporting  routines,  (4) 
Optimization  for  algorithm  execution  speed,  in  order  to 
meet  designated  latency  targets,  and  (5)  Implementation  of 
fallback  processing  in  case  inputs  are  missing.  A  single 
algorithm  generally  depends  on  several  inputs,  including 
fields  from  National  Center  for  Environmental  Prediction 
(NCEP),  as  well  as  output  fields  from  other  associated 
algorithms.  The  final  operational  version  of  the  algorithm 
generally  relies  on  the  integration  of  the  algorithm  with  an 
algorithm  chain  such  that  the  final  operational  version  of  the 
algorithm  may  be  fully  tested,  delivered,  and  validated. 

The  VIIRS  sensor  is  a  follow-on  instrument  to  the  United 
States’  National  Aeronautical  and  Space  Administration’s 
Moderate  Resolution  Imaging  Spectroradiometer  (MODIS) 
which  uses  36  spectral  bands  ranging  from  620  nm  to 
14.385  pm  [2],  while  VIIRS  uses  21  bands  ranging  from 
412  nm  to  11450  nm.  The  instruments  share  some 
similarities,  such  as  rotating  double-sided  mirror,  solar 
diffuser  and  solar  diffuser  stability  monitor,  which  create 
similarities  between  the  MODIS  and  VIIRS  Sensor  Data 
Record  (SDR)  processing  code,  for  calibration  and 
geolocation. 

II.  Overview  Of  V iirs  Science  To  Operational 
Algorithm  Conversion 

The  Visible/Infrared  Imager/Radiometer  Suite  (VIIRS) 
takes  measurements  within  the  visible  and  infrared  portion 
of  the  electromagnetic  spectrum.  The  VIIRS  instalment 
measures  reflectance’s,  brightness  temperatures,  and 
equivalent  black  body  temperatures  in  order  to  derive 
atmospheric,  land  and  water  surface  parameters  producing 
twenty-two  EDRs.  Of  these  22  EDRs,  the  VIIRS  is  the 
primary  instrument  for  eleven  EDRs  and  secondary  or 
contributory  to  eleven  others.  In  the  context  of  conversion 
from  science  to  operations  a  certain  number  of  the  EDRs 
pose  less  of  a  challenge  to  convert.  And,  conversely,  a 
number  pose  greater  challenges  to  convert  from  science 
grade  code  to  operational  code. 

Factors  contributing  to  the  conversion  complexity  include 
inter  sensor  mapping,  cross  granule  processing,  geolocation, 
ancillary  data,  intermediate  products,  algorithm  maturity, 
and  science  complexity.  Conversion  simplicity  for  more 
straight-forward  EDRs  is  characterized  by  fewer 
interdependencies  and  data  independence  from  other 
sensors.  Imagery  EDRs  using  the  visible  light  channels  fit 
this  category. 

More  complex  EDRs  include  sea  surface  temperature, 
cloud  base  height,  and  aerosol  optical  properties.  More 
complex  EDRs  may  require  multiple  steps  like  cloud 


4  The  AIX  (Unix)  platform  has  been  selected  by  the  NPOESS  program  as 
the  hosting  computer  for  the  Integrated  Data  Processing  System  portion  of 
the  Ground  Segment. 


clearing,  large  matrix  computations,  searching  Look  Up 
Tables  (LUTs)  generated  by  radiative  transfer  models,  etc. 

A.  Development  of  VIIRS  Science  Grade  Algorithm  Code 

The  VIIRS  Science  algorithms  were  developed  under  an 

independent  subcontract  to  the  government  by  Raytheon 
ITSS.  These  science  algorithms  included  SDR  code  for 
sensor  Calibration  and  Geolocation  as  well  as  the  EDR  code 
for  each  of  the  deliverable  EDR  products.  Test  data  for  the 
science  code  came  from  MODIS  (also  called  proxy  test 
data)  and  from  NGST’s  Integrated  Weather  Test  Bed 
Facility  (simulated  data).  Further  development  after  the 
science  code  was  delivered  to  NGST  included  integration  of 
Quality  Flags  (also  including  processing  flags,  identifying 
different  algorithm  branching  performed). 

Because  imagery  EDRs  are  created  directly  from  the 
VIIRS  Imagery  band  SDRs,  NGST  delivered  the  SDR  code 
with  technical  instructions  describing  which  map  projection 
to  use,  adjustments  for  sensor  visual  effects  such  as  “bow- 
tie,”  and  EDR  product  contents.  The  Day  Night  Band  on 
VIIRS  is  also  treated  as  imagery  SDR,  and  is  used  to  create 
the  Near  Constant  Contrast  EDR;  however,  a  science 
version  of  this  algorithm  was  delivered  for  conversion  to 
operational  code. 

Some  of  the  more  complex  EDR  algorithms  such  as  Cloud 
Base  Height  perform  physical  retrievals5,  where  the  cloud 
effective  particle  size  and  cloud  optical  thickness  are 
calculated,  temperature  profiles  are  used  to  estimate  lapse 
rates,  and  parallax  corrections  made,  in  order  to  calculate 
the  Cloud  Base  Height  EDR. 

B.  Conversion  Of  Science  Grade  Algorithm  Code  To 
Operational  Code 

The  five  step  conversion  process  from  science  to 
operational  code  is  characterized  by  two  Detailed  Design 
Peer  Reviews  (DDPRs),  first  at  the  beginning  of  the 
conversion  process  to  establish  common  Input-Processing- 
Output  architecture,  as  well  as  Error  Handling  and  Data 
Quality  Reporting,  and  a  second  DDPR  is  held  mid-way 
through  the  conversion  for  the  design  of  Graceful 
Degradation  (fallback  processing)  and  Optimization 
(latency  targeting).  These  two  DDPRs  are  held  for  each 
SDR  and  EDR  algorithm,  and  science  algorithm  leads  from 
the  sensor  vendor,  NGST,  and  the  science  community  is 
invited  to  participate  in  addition  to  the  operational  algorithm 
lead  and  ground  segment  science  leads.  After  each  of  the 
DDPRs  is  held  and  design  has  been  approved  and 
implemented,  a  Code  and  Unit  Test  (CUT)  Review  is  held 
to  vet  the  test  results,  confirming  that  the  conversion  is  not 
deviating  from  the  science  results.  There  is  one  CUT  for 
each  of  the  DDPRs. 


5 A  physical  retrieval  is  a  method  of  deriving  the  value  of  an  atmospheric 
variable  by  numerically  solving  the  radiative  transfer  equation  until 
calculated  radiances  match  observed  radiances  within  tolerance. 


At  each  stage  of  conversion,  after  the  DDPRs  and  CUTs, 
the  science  unit  test  data  delivered  with  the  algorithm  to 
Raytheon  are  run  through  the  operational  version  of  the 
code  on  the  IBM  AIX  platform,  and  formal  benchmark  test 
results  are  returned  to  NGST  for  verification  that  the 
operational  conversion  of  the  algorithm  is  within  pre¬ 
specified  accuracy,  precision,  and  uncertainty. 

Another  task  performed  during  the  conversion  process,  is 
to  populate  the  OAD  with  architecture  information, 
direction  from  NGST  regarding  checking  for  Quality  Flags 
and  Threshold  Tables,  and  to  describe  substitute  ancillary 
data  written  into  the  operational  version  of  the  science  code. 

Follow-on  tasks  described  by  NGST  in  Technical  Memos 
or  delivered  as  additional  science  code  are  incorporated  into 
the  operational  algorithm,  and  descriptions  of  these  changes 
are  added  to  the  OAD.  Issues  arising  out  of  algorithm  chain 


testing  and  integration  are  accommodated  through 
additional  reviews  and  testing. 

Finally,  the  conversion  of  the  algorithms  once  they  have 
been  unit  tested  proceeds  to  chain  testing  and  integration,  as 
well  as  a  more  robust  latency  analysis.  Two  of  the  most 
interdependent  algorithm  categories  include  the  cloud  and 
aerosol  algorithms. 

As  an  example,  the  testing  and  integration  of  the  Cloud 
algorithms  includes  two  types  of  chain  testing  and 
integration:  (1)  The  execution  of  the  Cloud  algorithm  chain 
itself  (in  order,  Cloud  Effective  Particle  Size,  Cloud  Optical 
Thickness,  Cloud  Top  Height,  Cloud  Top  Temperature, 
Cloud  Top  Pressure,  Cloud  Cover  and  Layers,  and  Cloud 
Base  Height),  and  (2)  The  dependency  of  some  of  the  Land 
algorithms  (Land  Albedo,  Land  Surface  Temperature,  and 
Vegetative  Index)  on  the  Cloud  algorithms  (Cloud  Optical 
Thickness,  Cloud  Effective  Particle  Size). 


m 


GRIDDING 


DAILY  SURF  REF 
MONTHLY  SURF  REF 
MONTHLY  BT 
MONTHLY  VI 
PREY.  ICE  AGE 
SNOW/ICE  CVR 


EE  [ 


GIP  GRANULATION 


PREV.  ICE  AGE 
QTRLY  SRF  TYPE 
SNOW/ICE  CVR 
Annual  MM  Monthly  VI 
Land  Albedo 
BRDF  Archetypal  File 


m 

m 


Sea  Ice  Characterization  EDR  (Not  in  EDR IR  vl  .7) 

Surface  Type,  Snow  Cover/Depth  EDR  (EDR  IR  vl  .7  adds  Active  Fires  *  Some  Graceful  Degradation  dependencies) 
Cloud  Mask  IP,  Sea  Surface  Temperature  EDR  (EDR  IR  vl  .7  does  not  have  input  for  SST) 

Snow  Cover/Depth  EDR  (in  EDR  IR  vl.7) 

Surface  Type  EDR  (in  EDR  IR  vl  .7) 

Surface  Albedo  EDR  (in  EDR  IR  vl  .7  inputs  to  Land  Albedo  IP  and  Snow  Cover/Depth  EDR) 

Surface  Albedo  EDR  (EDR  IR  vl.7  has  this  only  being  used  in  the  17  day  update  cycle  as  input  and  output) 


EE  f 


ANC  GRANULATION 


Legend 


O  GIP  to  be  Granulated  to  VIIRS 
^  Bundled  VIIRS  EDR  from  VIIRS  IPs 


Figure  1.  VIIRS  Wiring  Diagram  describing  the  operating  sequence  of  the  VIIRS  algorithms,  Intermediate  Products,  and  Raw,  Sensor,  and  Engineering  Data 


Records. 

III.  VlIRS  Sensor  Challenges 

A.  Challenges  in  Verification  of  VIIRS  Science  Grade 
Code  Performance 

Test  Data  of  two  types  has  been  used  for  unit  testing  of 
individual  algorithms:  (1)  Proxy  data  from  heritage 
instruments  and  (2)  simulated  data  from  radiative  transfer 
models.  In  the  case  where  proxy  data  is  available 
coincident  with  field  mission  data,  there  has  been  a 
preference  to  use  the  in  situ  data  collected  during  field 
missions  to  verify  proxy  data  from  heritage  instruments. 


CAMEX  field  data  was  used  for  comparison  purposes  with 
the  MODIS  cloud  algorithms6. 

Chain  testing  of  the  algorithms  will  consist  of  two  phases: 
(1)  Functional  testing  making  use  of  proxy  test  data  and 
ensuring  the  NPOESS  ground  system  includes  the  algorithm 
chain  functionality;  and  (2)  Performance  testing  to  make  use 
of  model  test  data  and  to  check  the  accuracy,  precision,  and 
uncertainty  of  each  of  the  EDRs. 

Both  Cloud  Optical  Thickness  and  Aerosol  Optical 
Thickness  have  a  considerable  number  of  downstream 


6  The  Convection  And  Moisture  Experiment  (CAMEX)  is  a  series  of  field 
research  investigations  sponsored  by  the  Earth  Science  Enterprise  of  the 
National  Aeronautics  and  Space  Administration  (NASA). 


dependencies  with  Land,  Ocean,  and  Surface  Temperature 
algorithms.  The  sensitivity  of  these  downstream  algorithms 
to  Optical  Thickness  values  is  a  difficult  attribute  to  capture, 
due  to  the  wide  variation  of  scene  conditions  covering  the 
globe  and  the  sequence  of  algorithm  availability.  Once  the 
full  chain  of  VIIRS  algorithms  has  been  integrated  and 
tested  with  a  global  test  data  set,  the  sensitivity  of  the 
downstream  algorithm  performance  will  be  measured. 

B.  Challenges  in  Migration  of  Science  Grade  Code  to 
Operational  Code 

A  challenge  in  development  of  algorithm  performance 
has  been  the  proliferations  of  Quality  Flags,  used  to  not  only 
identify  scene  conditions  affecting  a  measurement,  but  also 
sensor  threshold  states.  Specifically,  the  spatial  resolution 
of  the  VIIRS  instrument  causes  Quality  Flags  calculated  at 
the  pixel  level  to  balloon  the  storage  requirements  of  the 
SDR  and  EDR  products. 

The  Cloud  and  Aerosol  algorithm  have  a  significant 
number  of  downstream  dependencies  as  well  as  being  used 
to  identify  either  degraded  algorithm  performance  or 
conditions  where  the  algorithm  cannot  correctly  adapt  to  the 
physical  conditions  in  the  atmosphere,  and  at  which  point 
the  performance  is  excluded  from  skill  measurements.  An 
example  of  a  particular  Quality  Flag  that  translates  into 
flagging  such  pixels  is  for  Field  of  View  cases  where  the 
Aerosol  Optical  Thickness  exceeds  1.0.  [4]. 

An  example  of  where  Quality  Flags  endanger  product 
storage  requirements  of  proliferating  is  in  the  case  of  the 
V11RS  Cloud  Mask  Intermediate  Product.  This  product  is 
comprised  of  four  values  actually  describing  the  cloud  state 
(cloudy,  partially  cloudy,  clear,  and  partially  clear),  as  well 
as  twenty-eight  Quality  Flags  per  pixel  identifying  other 
scene  conditions  (cloud  phase,  presence  of  thin  cirrus,  fire, 
etc.).  Due  to  the  high  spatial  resolution,  wide  swath  width, 
and  high  data  volume  of  the  V11RS  products,  careful 
attention  has  been  given  to  the  management,  design,  and 
systems  engineering  of  VI1RS  Quality  Flags. 

IV.  Rewards  And  Hazards 

Benefits  of  having  an  operational  version  of  the  VIIRS 
algorithms  include  performance  optimization, 

modularization  to  facilitate  later  improvements, 
accommodating  unavailability  of  expected  input  data, 
allowing  for  graceful  degradation,  and  handling  of  recurring 
but  rare  orbital  events. 

A.  Rewards  of  VIIRS  Science  to  Operations  Conversion 


The  above  stated  benefits  are  derived  from  the  process  of 
integrating  the  algorithms  into  a  functioning,  interdependent 
chain.  This  process  includes  accounting  for  sensor 
hardware  behavior  in  the  SDR  algorithm,  and  taking  into 
account  the  behavior  of  the  sensor  in  the  context  of  the 
spacecraft  (thermal  effects,  orbital  maneuvers,  space 
environment). 

Because  the  individual  algorithms  have  generally  been 
developed  independently,  a  unified  operational  plan  has 
resulted  in  formalized  requirements  and  specifications  for  a 
systems  architecture. 

B.  Hazards  of  VIIRS  Science  to  Operations  Conversion 
The  treatment  of  many  of  the  science  algorithms  as  stand 
alone  calculations  allows  the  independent  verification  of 
each  algorithm's  performance.  However,  the  development 
and  delivery  to  the  Integrated  Data  Processing  Segment  of 
the  EDR  algorithms  prior  to  completion  and  testing  of  the 
SDRs,  has  complicated  the  procedures  used  for  Algorithm 
Chain  Testing. 

V.  Summary 

This  paper  addressed  selected  elements  of  the  process 
being  used  to  convert  and  test  the  maturing  and  changing 
VIIRS  science  grade  algorithm  code  to  the  initial 
operational  algorithm  source  code.  The  paper  described 
VIIRS  EDRs  that  were  particularly  vexing  and  challenging 
to  convert.  Many  reviews  are  performed  during  the 
conversion  in  order  to  maintain  the  integrity  of  the  original 
science  code. 

It  may  benefit  the  scientists  to  become  familiar  with  all 
aspects  of  this  process,  and,  especially  the  modular  form  (I- 
P-O)  of  the  original  science  code.  Running  this  code  on 
their  own  platforms  will  facilitate  their  familiarity  with  the 
benefits  and  features  of  operational  science  code,  diagnosis 
of  on-orbit  anomalies,  code  maintenance  and  future 
improvements.  Using  the  operational  version  of  the  science 
code  will  also  facilitate  future  algorithm  improvement 
development  and  minimize  the  time  for  testing,  verification, 
and  validation,  and  configuration  control  for  operational 
implementation  of  improved  algorithms. 
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